Decison Boundary and hyperbolic unfolding

Decision boundary of a neural network

Dataset with two classes

Train a neural network on the binary classification task

# define fully connected NN architecture
circle_detect = Net(0, [2,10,20])


# Train neural net on binary classification task
_ = train_classification_nn(circle_detect, data_circ, label_circ, n_epochs=15)

epoch	train_loss	valid_loss	accuracy	time
0	0.814834	0.792440	0.516000	00:01
1	0.796542	0.750172	0.516000	00:00
2	0.573485	0.481312	0.977000	00:00
3	0.398306	0.355876	1.000000	00:00
4	0.338907	0.327376	1.000000	00:00
5	0.322854	0.319389	1.000000	00:00
6	0.317590	0.316259	1.000000	00:00
7	0.315475	0.314866	1.000000	00:00
8	0.314508	0.314196	1.000000	00:00
9	0.314028	0.313856	1.000000	00:00
10	0.313778	0.313679	1.000000	00:00
11	0.313646	0.313585	1.000000	00:00
12	0.313577	0.313540	1.000000	00:00
13	0.313546	0.313523	1.000000	00:00
14	0.313538	0.313520	1.000000	00:00

Contour plot of the neural network function

The decision boundary is $\left\{x\in \mathbb R^2:\ nn(x)=\frac 12\right\}$, i.e. all points where the output of the neural network is ambiguous.

Ways to compute the decision boundary

First Approach: Sampling algorithm

• Using methods described in "On The Topological Expressive Power of Neural Networks", brought to our attention by Stefania at the last AppTop Meet-up.
• Implementation by Matteo.

Gradient flow method

• We are pushing points to the decision boundary.
• We are using automatic differentiation of neural networks.
• The algorithm is fully implemented in Pytorch; hence it can be enormously accelerated via GPUs.

Sample points $x_i$ with gradients $-\nabla_{x_i} \left(nn(x_i)-\frac 12\right)^2$.

After 50 iterations and a filtration with respect to $|nn(x_i)-0.5|>0.01$ we end up with the red points:

Comparision with the previously plotted decision boundary.

Hyperbolic unfolding

(joint work with Matthias Kemper)

• We transform the euclidian metric conformally with factor $1/(nn-\frac 12)$.
• The resulting metric space is (Gromov-)hyperbolic with the decision boundary as the boundary at infinity.
• Geodesic rays converge to the boundary at infinity.
• We use numerical methods for solving geodesic equations (second-order differential equations).

Computed decision boundary with dataset

Entanglement problem for tori

Decision boundary for entangled tori

Decision boundary for unentangled tori